Neurobiology of Language

In preschool years, children take important steps in grammar acquisition, which are essential to learning their native language. A central aspect is the acquisition of the morpho-syntactic rule system, which forms an intersection between words and sentences. In adults, rule-based linguistic processes are supported by the dorsal fiber pathway to BA44, the arcuate fascicle. This pathway matures relatively late in development, raising the question of whether it already supports grammar processes in the early preschool years, or whether early grammar acquisition is supported by different, earlier-maturing fiber pathways. In two independent samples of 3- to 5-year-old children (N = 90 and N = 30), we examined the association between the maturation of fiber pathways of the language network and childrens noun plural assignment as an index of their morpho-syntactic abilities. This revealed consistent differences between 3-year-olds and 4- to 5-year-olds. The 4- and 5-year-olds, but not 3-year-olds, showed a relation of morpho-syntax with both the dorsal pathway to BA44, supporting syntactic processes, and the dorsal pathway to BA6, supporting phonological processes in adults. Our results suggest that, in contrast to adults, preschool-aged children rely on both dorsal fiber pathways for morpho-syntax. This difference might point to different processing strategies reflecting the transition from phonology-based statistical learning to rule-based learning in grammar acquisition.

Recent large language models (LLMs) have demonstrated remarkable proficiency in complex linguistic tasks and have been shown to share certain computational principles with human language processing. However, whether LLMs internal components perform distinct functions, like semantic and syntactic processing in human language systems, remains unclear. Here, we systematically disrupted components of LLMs to simulate the behavioral profiles of aphasia--a disorder characterized by specific language deficits resulting from brain injury. Our findings showed that lesioning specific components of LLMs could replicate behaviors characteristic of different aphasia subtypes. Notably, while semantic deficits as those observed in Wernickes and Conduction aphasia, were relatively straightforward to simulate, reproducing syntactic and lexical impairments, as seen in Brocas and Anomic aphasia, proved more challenging. Together, these results highlight both parallels and discrepancies between emergent modularity in LLMs and the human language system, providing new insights into how information is represented and processed in artificial and biological intelligence.

Studies of perception have long shown that the brain adds information to its sensory analysis of the physical environment. A touchstone example for humans is language use: to comprehend a physical signal like speech, the brain must add linguistic knowledge, including syntax. Yet, syntactic rules and representations are atemporal (i.e., abstract and not bound by time), so they must be translated into time-varying signals for speech comprehension and production. Here, we test three different models of the temporal spell-out of syntactic structure against brain activity of people listening to Dutch stories: an integratory bottom-up parser, a predictive top-down parser, and a mildly predictive left-corner parser. These models build exactly the same structure but differ in when syntactic information is added by the brain - this difference is captured in the (temporal distribution of the) complexity metric incremental node count. Using temporal response function models with both acoustic and information-theoretic control predictors, node counts were regressed against source-reconstructed delta-band activity acquired with magnetoencephalography. Neural dynamics in left frontal and temporal regions most strongly reflect node counts derived by the top-down method, which postulates syntax early in time, suggesting that predictive structure building is an important component of Dutch sentence comprehension. The absence of strong effects of the left-corner model further suggests that its mildly predictive strategy does not represent Dutch language comprehension well, in contrast to what has been found for English. Understanding when the brain projects its knowledge of syntax onto speech, and whether this is done in language-specific ways, will inform and constrain the development of mechanistic models of syntactic-structure building in the brain.

Recent studies have demonstrated that fine-tuning language models with brain data can improve their semantic understanding, although these findings have so far been limited to English. Interestingly, similar to the shared multilingual embedding space of pretrained multilingual language models, human studies provide strong evidence for a shared semantic system in bilingual individuals. Here, we investigate whether fine-tuning language models with bilingual brain data changes model representations in a way that improves them across multiple languages. To test this, we fine-tune monolingual and multilingual language models using brain activity recorded while bilingual participants read stories in English and Chinese. We then evaluate how well these representations generalize to the bilingual participants first language, their second language, and several other languages that the participants are not fluent in. We assess the fine-tuned language models on brain encoding performance and downstream NLP tasks. Our results show that bilingual brain-informed fine-tuned language models outperform their vanilla (pretrained) counterparts in both brain encoding performance and most downstream NLP tasks across multiple languages. These findings suggest that brain-informed fine-tuning improves multilingual understanding in language models, offering a bridge between cognitive neuroscience and NLP research. We make our code publicly available. 2

This study investigates the neural dynamics of phoneme processing in 7-year-old children with and without dyslexia (25;9 [male]), using EEG recordings collected during continuous speech listening. By applying temporal generalization to phonetic descriptor decoding, we can disentangle whether potential phoneme processing deficits are due to the maintenance of phonemes in verbal short-term memory and/or inferred differences in phonetic processing speed, both of which are thought to be impaired in dyslexia. We investigated whether phonetic processing depends on the phonemes position or its lexical competition. Our results reveal two key findings that may help explain the challenges faced by children with dyslexia. First, these children exhibit reduced decoding accuracy for word-onset phonemes, suggesting disruptions in either predictive, word-level anticipatory mechanisms or in the intrinsic rhythmic processing aligned with word boundaries. Second, they exhibit increased decoding accuracy for non-onset phonemes with low lexical competition approximately 400 ms after phoneme onset. This pattern suggests that children with dyslexia retain linguistically less relevant sounds longer in verbal short-term memory and process them more slowly compared to their typical reading peers. Together, these findings suggest that dyslexia is characterized by altered phonetic encoding strategies, specifically inefficient prioritization of relevant phonological information. This work provides new insight into the neural mechanisms underlying phonological deficits and contributes to a deeper understanding of the cognitive basis of dyslexia. Significance statementDyslexia is associated with difficulties in phonological processing. Investigating EEG during continuous speech listening, we show that children with dyslexia exhibit weaker encoding of word-onset phonemes and prolonged processing of less informative phonemes. These altered encoding strategies suggest inefficient prioritization of linguistic information, offering new insight into the neural basis of dyslexia.

A network of left frontal and temporal brain regions supports high-level language processing-- including the processing of word meanings, as well as word-combinatorial processing--across presentation modalities. This core language network has been argued to store our knowledge of words and constructions as well as constraints on how those combine to form sentences. However, our linguistic knowledge additionally includes information about sounds (phonemes) and how they combine to form clusters, syllables, and words. Is this knowledge of phoneme combinatorics also represented in these language regions? Across five fMRI experiments, we investigated the sensitivity of high-level language processing brain regions to sub-lexical linguistic sound patterns by examining responses to diverse nonwords--sequences of sounds/letters that do not constitute real words (e.g., punes, silory, flope). We establish robust responses in the language network to visually (Experiment 1a, n=605) and auditorily (Experiments 1b, n=12, and 1c, n=13) presented nonwords relative to baseline. In Experiment 2 (n=16), we find stronger responses to nonwords that obey the phoneme-combinatorial constraints of English. Finally, in Experiment 3 (n=14) and a post-hoc analysis of Experiment 2, we provide suggestive evidence that the responses in Experiments 1 and 2 are not due to the activation of real words that share some phonology with the nonwords. The results suggest that knowledge of phoneme combinatorics and representations of sub-lexical linguistic sound patterns are stored within the same fronto-temporal network that stores higher-level linguistic knowledge and supports word and sentence comprehension.

Transformer-based large language models (LLMs) have significantly advanced our understanding of meaning representation in the human brain. However, increasingly large LLMs have been questioned as valid cognitive models due to their extensive training data and their ability to access context thousands of words long. In this study, we investigated whether instruction tuning, another core technique in recent LLMs beyond mere scaling, can enhance models ability to capture linguistic information in the human brain. We evaluated the self-attention of base and fine-tuned LLMs of different sizes against human eye movement and functional magnetic resonance imaging (fMRI) activity patterns during naturalistic reading. We show that scaling has a greater impact than instruction tuning on model-brain alignment, reinforcing the scaling law in brain encoding performance. These finding have significant implications for understanding the cognitive plausibility of LLMs and their role in studying naturalistic language comprehension.

Human language processing involves not only combining word meanings in accordance with semantic and syntactic constraints, but also figuring out who and what is being referred to. Here we present a first study towards a mechanistic understanding of the neural basis for referential processing. Using both functional MRI and magnetoencephalography (MEG), we identified a consistent increase of activity in a network spanning the anterior and posterior left middle temporal gyrus and the angular gyrus for pronoun processing during naturalistic listening for both English and Chinese speakers. We then adopted a "reverse-engineering" approach to examine the cognitive processes underlying pronoun resolution. We evaluated the neural fit of three symbolic models that each formalizes a different strand of explanation for pronoun resolution in the cognitive and linguistic literature, as well as two deep neural network models with an LSTM or a Transformer architecture. Our results favor the memory-based symbolic model, suggesting a domain-general mechanism of pronoun resolution that resembles memory retrieval.

Dyslexia is a neurobiological disorder characterised by reading difficulties, yet its underlying causes remain unclear. Neuroimaging and behavioural studies found anomalous responses in tasks requiring phonological processing, motion perception, and implicit learning, and showed gray and white matter abnormalities in several brain regions of dyslexics compared to controls, indicating that dyslexia is a heterogeneous condition and promoting a multifactorial approach. In order to evaluate whether the combination of behavioural and multimodal MRI can have greater sensitivity in identifying neurocognitive traits of dyslexia compared to monocomponential approaches, in 19 dyslexic and 19 control subjects we acquired behavioural cognitive assessments, multiple (phonological, visual motion, rhythmic) mismatch-response functional MRI tasks, structural diffusion-weighted and T1-weighted images. To examine between-group differences in the multimodal neurocognitive measures, we applied univariate and multivariate approaches. Results showed that dyslexics performed worse than controls in behavioural phonological tasks. Neuroimaging analyses revealed that individuals with dyslexia present reduced cerebellar responses to mismatching rhythmic stimuli, as well as structural disorganization in several white matter tracts and cortical regions previously implicated in dyslexia. Most importantly, in line with the view of dyslexia as a multifactorial phenomenon, a machine learning model trained with features from all three MRI modalities (functional, diffusion, and T1-weighted) discriminated between dyslexics and controls with greater accuracy than models including just one modality. The individual classification scores in the multimodal machine learning model correlated with behavioural reading accuracy. These results confirm that dyslexia should be approached as a composite condition characterised by multiple distinctive cognitive and brain features.

Bilingualism research has long been challenged by a lack of a unified approach to quantifying language dominance and degree of multilingualism. While numerous questionnaires (e.g., LHQ, BLP, LEAP-Q, and LUQ) provide valuable data on language background variables, they lack a standardized formula to compute key measures from it. We introduce two formulas that synthesize critical linguistic variables to efficiently calculate language dominance and a multilingualism score that ranges from perfect monolingualism to native-like proficiency in multiple languages. Validation across two large datasets shows our dominance measure closely aligns with more complex PCA methods while being simpler and more efficient.

When a person listens to natural speech, the relation between features of the speech signal and the corresponding evoked electroencephalogram (EEG) is indicative of neural processing of the speech signal. Using linguistic representations of speech, we investigate the differences in neural processing between speech in a native and foreign language that is not understood. We conducted experiments using three stimuli: a comprehensible language, an incomprehensible language, and randomly shuffled words from a comprehensible language, while recording the EEG signal of native Dutch-speaking participants. We modeled the neural tracking of linguistic features of the speech signals using a deep-learning model in a match-mismatch task that relates EEG signals to speech, while accounting for lexical segmentation features reflecting acoustic processing. The deep learning model effectively classifies languages. We also observed significant differences in tracking patterns between comprehensible and incomprehensible speech stimuli within the same language. It demonstrates the potential of deep learning frameworks in measuring speech understanding objectively.

At first glance, the brains language network appears to be universal, but languages clearly differ. How does the brain adapt to the specific details of individual grammatical systems? Here, we present an MEG study on case and agreement in Hindi and Nepali. Both languages use split-ergative case systems. However, these systems interact with verb agreement differently - in Hindi, case features conspire to determine which noun phrase (NP) the verb agrees with (subject, object, or neither), but in Nepali the verb always agrees with the subject NP. We found that left inferior frontal and left anterior temporal regions are sensitive to case features in both languages. Across case configurations, these same brain areas in Hindi participants show different patterns of activity for sentences that require masculine vs. feminine marking on the verb, before the comprehenders encounter it. Additionally, the left temporoparietal junction in Hindi shows different activity for subject and object agreement configurations. Both findings are not observed in Nepali participants. We suggest that this brain response demonstrates a unique-to-Hindi selection of an agreement controller and pre-encoding of the verbs morphological features. This shows that brain activity reflects psycholinguistic processes that are intimately tied to grammatical features. HighlightsO_LIThe left inferior frontal lobe and the left anterior temporal lobe distinguish accusative objects versus bare object NPs in Hindi and Nepali, and pre-emptively encode gender agreement features in Hindi. C_LIO_LIThe left inferior parietal lobe shows a differential sensitivity to object-agreement and subject-agreement constructions in Hindi that is absent in Nepali C_LIO_LIMEG can reveal differences in neural activity that reflect specific requirements of different grammatical systems C_LI

The use of multiple languages modulates the neural mechanisms of selective attention, but it is unclear whether these adaptations require continuous engagement and active second language usage. Here we examined whether language usage shapes selective attention in bilingualism, and what neural processes might this engage. 48 highly proficient English-French bilinguals listened to naturalistic speech streams in their first or second language, paired with either linguistic and non-linguistic interference. Participants were matched on their L2 proficiency, but were either Active, Moderate, or Inactive users of their second language. The results revealed usage-related modulation of oscillatory activity in the alpha band, with more efficient inhibitory control in Active and Moderate users leading to behavioural resilience to interference. In contrast, usage did not affect lower-level perceptual tracking of speech, as captured by mTRF decoding of speech envelopes across frequency bands. Taken together, our findings show that resilience to interference during language processing is not dependent on the perceptual speech tracking, but rather on the capacity to recruit higher-level attentional control mechanisms, a process that is dynamically shaped by bilinguals L2 usage.

Predicting upcoming events is critical to our ability to effectively interact with our environment and conspecifics. In natural language processing, transformer models, which are trained on next-word prediction, appear to construct a general-purpose representation of language that can support diverse downstream tasks. However, we still lack an understanding of how a predictive objective shapes such representations. Inspired by recent work in vision neuroscience Henaff et al. (2019), here we test a hypothesis about predictive representations of autoregressive transformer models. In particular, we test whether the neural trajectory of a sequence of words in a sentence becomes progressively more straight as it passes through the layers of the network. The key insight behind this hypothesis is that straighter trajectories should facilitate prediction via linear extrapolation. We quantify straightness using a 1-dimensional curvature metric, and present four findings in support of the trajectory straightening hypothesis: i) In trained models, the curvature progressively decreases from the first to the middle layers of the network. ii) Models that perform better on the next-word prediction objective, including larger models and models trained on larger datasets, exhibit greater decreases in curvature, suggesting that this improved ability to straighten sentence neural trajectories may be the underlying driver of better language modeling performance. iii) Given the same linguistic context, the sequences that are generated by the model have lower curvature than the ground truth (the actual continuations observed in a language corpus), suggesting that the model favors straighter trajectories for making predictions. iv) A consistent relationship holds between the average curvature and the average surprisal of sentences in the middle layers of models, such that sentences with straighter neural trajectories also have lower surprisal. Importantly, untrained models dont exhibit these behaviors. In tandem, these results support the trajectory straightening hypothesis and provide a possible mechanism for how the geometry of the internal representations of autoregressive models supports next word prediction.

Infants and adults show the remarkable ability to learn from statistical regularities in the environment. Seminal studies in language acquisition suggested that transitional probabilities between syllables are decisive for language learning. Yet, recent work cautioned that acoustic and phonological regularities can confound transitional probabilities, compromising interpretability. Furthermore, prior linguistic background can impact the learning of a new (artificial) language. To control for such confounds, we developed an open-source Python toolbox that generates Artificial Languages with Phonological and Acoustic Rhythmicity Controls (APLARC). First, we explain all functionalities of ALPARC through a step-by-step guide. Then, we demonstrate how ALPARC generates syllable streams encompassing pseudowords that are tailored to critical statistics of real languages. Our results show that ALPARC streams attain more stationary transitional probability distributions and minimize phonological and acoustic confounds relative to stimuli used in prior studies. We conclude that ALPARC would greatly boost the interpretability of future language learning research.

A critical way for humans to acquire, represent and communicate information is through language, yet the underlying computation mechanisms through which language contributes to our word meaning representations are poorly understood. We compared three major types of word computation mechanisms from large language corpus (simple co-occurrence, graph-space relations and neural-network-vector-embedding relations) in terms of the association of words brain activity patterns, measured by two functional magnetic resonance imaging (fMRI) experiments. Word relations derived from a graph-space representation, and not neural-network-vector-embedding, had unique explanatory power for the neural activity patterns in brain regions that have been shown to be particularly sensitive to language processes, including the anterior temporal lobe (capturing graph-common-neighbors), inferior frontal gyrus, and posterior middle/inferior temporal gyrus (capturing graph-shortest-path). These results were robust across different window sizes and graph sizes and were relatively specific to language inputs. These findings highlight the role of cumulative language inputs in organizing word meaning neural representations and provide a mathematical model to explain how different brain regions capture different types of language-derived information.

At the heart of language neuroscience lies a fundamental question: How does the brain process the rich variety of languages? Multilingual neural network models offer a way to answer this question by representing linguistic content across languages in a shared space. Leveraging these advances, we evaluated the similarity of linguistic representations in speakers of 21 languages. We combined existing (12 languages across 4 language families) and newly collected fMRI data (9 languages across 4 families) to test encoding models predicting brain activity in the language network using representations from multilingual models. Model representations reliably predicted brain responses within each language. Critically, encoding models can be transferred zero-shot across languages, so that a model trained to predict brain activity in a set of languages can account for responses in a held-out language. These results imply a shared cross-lingual component, which appears to be related to a shared meaning space.

Most neuroimaging studies of speech disfluency have compared individuals who stutter with fluent controls. However, treating speech disfluency as a continuous, dimensional trait offers new insights into the neural basis of fluency during early childhood. This study aimed to investigate whether naturally occurring variation in speech disfluency is associated with grey matter structure in a non-clinical, population-based sample of 5-year-old children. The study included 120 participants (65M, 55F) from the FinnBrain Birth Cohort study. Speech disfluency was evaluated as a continuous measure from audiovisual speech samples, with transcription and analysis conducted using the SALT software. Ambrose & Yairi (1999) classification system was used to categorize speech disfluencies into stuttering-like (SLD) and other disfluency types. T1-weighted images obtained through magnetic resonance imaging were analyzed using voxel-based morphometry (VBM) with the CAT12 toolbox and complemented by surface-based morphometry with FreeSurfer. Whole-brain statistical analysis was employed to examine the association between grey matter metrics and speech disfluency. We found that VBM-derived proportional grey matter volume in the left middle frontal gyrus, left posterior cerebellum, and right superior frontal gyrus was positively associated with speech disfluency, specifically SLD, in children (p <.001; p=.002; p <.001, FDR corrected). No significant associations were found for cortical thickness or surface area. Additionally, no notable sex differences were observed. Our findings suggest that speech disfluency in early childhood is linked to localized structural differences in regions supporting motor planning and cognitive control, without broader changes in cortical thickness or surface area. Importantly, similar brain regions have been implicated in studies comparing children who stutter to those who do not, suggesting that normal variation in disfluency captures meaningful neurobiological differences even in non-clinical populations. This supports the value of treating speech disfluency as a spectrum and underscores the importance of longitudinal, multimodal research to clarify how these structural features evolve and influence later fluency outcomes.

How do polyglots--individuals who speak five or more languages--process their languages, and what can this population tell us about the language system? Using fMRI, we identified the language network in each of 34 polyglots (including 16 hyperpolyglots with knowledge of 10+ languages) and examined its response to the native language, non-native languages of varying proficiency, and unfamiliar languages. All language conditions engaged all areas of the language network relative to a control condition. Languages that participants rated as higher-proficiency elicited stronger responses, except for the native language, which elicited a similar or lower response than a non-native language of similar proficiency. Furthermore, unfamiliar languages that were typologically related to the participants high-to-moderate-proficiency languages elicited a stronger response than unfamiliar unrelated languages. The results suggest that the language networks response magnitude scales with the degree of engagement of linguistic computations (e.g., related to lexical access and syntactic-structure building). We also replicated a prior finding of weaker responses to native language in polyglots than non-polyglot bilinguals. These results contribute to our understanding of how multiple languages co-exist within a single brain and provide new evidence that the language network responds more strongly to stimuli that more fully engage linguistic computations.

When listening to speech in their native language, speakers use prior context to anticipate upcoming phonemes, words, and concepts, integrating information at the sublexical, lexical, and sentence level. While it has been suggested that late second language learners do not predict to the same extent as native listeners, adequately evaluating this claim requires measurement of predictions at these multiple levels of representation simultaneously in natural speech. We recorded magnetoencephalography (MEG) responses from native Mandarin and Sinhala speakers listening to continuous narrative English speech. We used multivariate temporal response function (mTRF) analysis to investigate whether second language listeners demonstrate the same markers of prediction in neural data as native English speakers listening to the same stimuli. We demonstrate that late second language listeners exhibit strikingly similar responses to native speakers in sensitivity to phoneme surprisal and entropy with respect to sublexical, lexical, and sentence-level context. The few small response differences we observed appear most likely to arise from specific properties of the native languages, rather than general differences between native and second-language listening. These results provide evidence that late second-language listeners indeed leverage prediction in similar ways as native listeners in understanding continuous speech. This suggests that multivariate analyses of neural data from naturalistic listening may be vital in carefully evaluating the differences and similarities in speech prediction across populations. Significance StatementMuch is still unknown about how people listening to a second language predict upcoming words and sounds. Here, we leverage neuroimaging during continuous speech and analyze responses to multiple speech language features in the signal to study the neural encoding of prediction simultaneously at multiple levels of linguistic context. We observe robust encoding of statistical properties tied to prediction at all context levels in second-language learners of English and that responses are strikingly similar between native and second language listeners. Speech language features are encoded similarly in both groups of language learners, with few differences between the native and second language listeners, indicating that second language listeners predict upcoming input similarly to native listeners.